We consider the problem of building a reinforcement learning (RL) agent that can both accomplish non-trivial tasks, like winning a real-time strategy game, and strictly follow high-level language commands from humans, like "attack", even if a command is sub-optimal. We call this novel yet important problem, Grounded Reinforcement Learning (GRL). Compared with other language grounding tasks, GRL is particularly non-trivial and cannot be simply solved by pure RL or behavior cloning (BC). From the RL perspective, it is extremely challenging to derive a precise reward function for human preferences since the commands are abstract and the valid behaviors are highly complicated and multi-modal. From the BC perspective, it is impossible to obtain perfect demonstrations since human strategies in complex games are typically sub-optimal. We tackle GRL via a simple, tractable, and practical constrained RL objective and develop an iterative RL algorithm, REinforced demonstration Distillation (RED), to obtain a strong GRL policy. We evaluate the policies derived by RED, BC and pure RL methods on a simplified real-time strategy game, MiniRTS. Experiment results and human studies show that the RED policy is able to consistently follow human commands and achieve a higher win rate than the baselines. We release our code and present more examples at https://sites.google.com/view/grounded-rl.

In this paper, we propose ELF, an Extensive, Lightweight and Flexible platform for fundamental reinforcement learning research. Using ELF, we implement a highly customizable real-time strategy (RTS) engine with three game environments (Mini-RTS, Capture the Flag and Tower Defense). Mini-RTS, as a miniature version of StarCraft, captures key game dynamics and runs at 165K frame-per-second (FPS) on a laptop. When coupled with modern reinforcement learning methods, the system can train a full-game bot against built-in AIs end-to-end in one day with 6 CPUs and 1 GPU. In addition, our platform is flexible in terms of environment-agent communication topologies, choices of RL methods, changes in game parameters, and can host existing C/C++-based game environments like ALE. Using ELF, we thoroughly explore training parameters and show that a network with Leaky ReLU and Batch Normalization coupled with long-horizon training and progressive curriculum beats the rule-based built-in AI more than 70% of the time in the full game of Mini-RTS. Strong performance is also achieved on the other two games. In game replays, we show our agents learn interesting strategies.

We consider the problem of building a reinforcement learning (RL) agent that can both accomplish non-trivial tasks, like winning a real-time strategy game, and strictly follow high-level language commands from humans, like "attack", even if a command is sub-optimal. We call this novel yet important problem, Grounded Reinforcement Learning (GRL). Compared with other language grounding tasks, GRL is particularly non-trivial and cannot be simply solved by pure RL or behavior cloning (BC). From the RL perspective, it is extremely challenging to derive a precise reward function for human preferences since the commands are abstract and the valid behaviors are highly complicated and multi-modal. From the BC perspective, it is impossible to obtain perfect demonstrations since human strategies in complex games are typically sub-optimal.

The main proposal of the paper is a real-time strategy simulator specifically designed for reinforcement learning purposes. The paper presents with several details the architecture of the simulator, along with how gaming is done on it and some experimentations with the software with some RL techniques implemented in the software. Although I think there are good values in making with software for research, I don't think that NIPS is the right forum for presenting technical papers on them. Machine Learning Open Source Software (MLOSS) track from JMLR or relevant workshop are much relevant for that. And in the current case, a publication in the IEEE Computational Intelligence and Games (IEEE-CIG) conference might be a much better fit.

This paper focuses on real-time strategy games, and presents a model to make predictions over the parts of the game state that are not observable, as well as predicting the evolution of the game state over time. The proposed model is based on a encoder/decoder architecture that integrates convolutional networks with recurrent neural networks. This is an interesting paper with promising results. The most interesting part for me is that the proposed model seems to me a "starting point", and this opens up a very interesting avenue of research for the future. For example, the current module provides a prediction, but could it be used to provide a distribution over the possible game states, from where we can sample?

Last April, in that small glass conference room, the studio's decades of experience were marshaled in service of figuring out what Frost Giant's debut game would be called. Around the room, sheets of paper held up by blue tape were marked with key pillars of a good video game title, including "Game Fit," "Cool Factor," and for Frost Giant's debut specifically, "Hopeful/Optimistic." Tim Morten, the studio's CEO and co-founder, quietly recused himself from the selection process. Broad-shouldered and bearing a thin, near-constant smile, Morten was a selling point for the studio; some Frost Giant employees defected from high-ranking positions elsewhere to work with him. When the meeting began, he agreed to take notes.

We present CLASSQL, a multi-agent model for playing real-time strategy games, where learning and control of our own team's units is decentralized; each agent uses its own reinforcement learning process to learn and control units of the same class. Coordination between these agents occurs as a result of a common reward function shared by all agents and synergistic relations in a carefully crafted state and action model for each class.

Real-time strategy is having a moment. And gaming's largest companies, including Microsoft and Tencent, are bankrolling studios behind new RTS entries like Age of Empires IV, which is set for release on October 28. This resurgence is good news for fans of real-time strategy games, but the genre must adapt to tastes of modern gamers. Fortunately, the developers behind tomorrow's blockbuster real-time strategy games are mindful of the genre's past mistakes. The seed of the real-time strategy genre was planted when Chris Crawford published a treatise on the future of real-time gaming, titled "The Future of Computer Wargaming," in the debut winter 1981 issue of Computer Gaming World.

Stratega, a general strategy games framework, has been designed to foster research on computational intelligence for strategy games. In contrast to other strategy game frameworks, Stratega allows to create a wide variety of turn-based and real-time strategy games using a common API for agent development. While the current version supports the development of turn-based strategy games and agents, we will add support for real-time strategy games in future updates. Flexibility is achieved by utilising YAML-files to configure tiles, units, actions, and levels. Therefore, the user can design and run a variety of games to test developed agents without specifically adjusting it to the game being generated. The framework has been built with a focus of statistical forward planning (SFP) agents. For this purpose, agents can access and modify game-states and use the forward model to simulate the outcome of their actions. While SFP agents have shown great flexibility in general game-playing, their performance is limited in case of complex state and action-spaces. Finally, we hope that the development of this framework and its respective agents helps to better understand the complex decision-making process in strategy games. Stratega can be downloaded at: https://github.research.its.qmul.ac.uk/eecsgameai/Stratega